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ITEM TRACKING AND PROCESSING SYSTEMS AND METHODS 

CROSS-REFERENCE TO RELATED APPLICATIONS 
[0001] This application claims the benefit of U.S. Provisional Application No. 60/45 1 ,999, 
15 filed March 4, 2003, which is hereby fully incorporated herein in its entirety and made a part 
hereof. 

BACKGROUND OF THE INVENTION 
[0002] Field of the Invention 
20 [0003] The field of the present invention includes the tracking and processing of items. In 
particular, the present invention involves the communication of sorting instructions to a person 
during the processing of parcels. 
[0004] Description of Related Art 

[0005] The manual sorting or item-processing environment is readily described as a wide 
25 range of event-based stimuli with physical dynamic activity. For example, the current state of 
parcel processing is one where people who process parcels within a manual sorting facility are 
continually reading package information from each package's label. Given the acquired 
information, a range of decision types and activity are possible for each job type (the "per- 
package decision process"). Items are moved between job positions in sorting facilities using a 
30 flexible array of conveyor belts, slides, trays, bags, carts, etc. Large-scale item processors, such 
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as for example, UPS, have a substantial investment in the numerous facilities, plant equipment 
configurations, and training needed to provide the current state of the process. 
[0006] Any attempt to use technology to aid the per-item decision process is hampered by 
the high cost of inserting technology into existing manual package-processing environments. 
5 Challenges with the use of technology are also present in the form of space constraints as well as 
the flow of items in a processing environment. 

[0007] The biggest cost impacts of technology insertion are in providing stations to 
electronically acquire or read item data and providing stations to display or generate item sorting 
and/or processing instructions. The difficulty in minimizing these costs is that the accumulated 

10 exception rates for item processing is often very high. Factors that contribute to this exception 
rate include errors in conventional label codes scanning, address validation problems, package 
data availability, and package dimensional conformity. Therefore, a large expense is incurred in 
item processing by the need and processes of exception handling capabilities. 
[0008] Many conventional item-processing systems utilize on-the-floor item processing 

15 exception areas where an exception item is physically removed from the processing system and 
handled on an expensive and labor intensive individual basis. These on-the-floor areas may 
adversely impact the processing facility's balance of facility configuration, productivity, methods 
and throughput. 

[0009] In some instances, off-the-floor exception handling may be able to reduce physical 
20 exception handling. These systems may use item acquire and re-acquire stations whereby 
instances of label acquisition exceptions and instruction-change exceptions are handled 
electronically rather than manually. However, the use of off-the-floor exception areas enabled 
by fixed item acquire and re-acquire stations imposes an early processing deadline and does not 
allow for instruction changes after an item has passed the re-acquire station. Also, this method 
25 still requires considerable on-the-floor equipment for both, acquire and re-acquire stations. 

[0010] Embodiments of the present invention overcome many of the challenges present in 
the art, some of which are presented above. 
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BRIEF SUMMARY OF THE INVENTIONS 
[0011] Embodiments of the present invention provide computer-assisted decision 

capability for the processing of items. In a specific application, an embodiment of the present 
invention tracks and provides processing instructions for items within an item processing 
5 facility's handling processes. 

[0012] In other embodiments, items are tracked and information about one or more items is 
provided to a person based on the location of the person and/or the location of the one or more 
items. 

[0013] Generally, an embodiment of the invention involves a system whereby item handling 
10 personnel and supervisors wear a set of see-through display lenses that superimpose relevant 
messages proximately about or over real tracked objects in the field of view. These lenses are 
attached to an information gathering device that captures and decodes information about the item 
such as, for example, label images, and an orientation and position device that determines the 
orientation and position of the wearer so that it may be determined what items are in the field of 
15 view. 

[0014] Embodiments of the present invention involve a data acquisition and display device 
comprised of an information gathering device to capture data from an object, a beacon detection 
device to capture information about the orientation and position of a wearer, and a transparent 
heads-up display showing instructions related to the object, each in communication with one or 
20 more computers. 

[0015] Another aspect of the present invention is a tracking system such as, for example, an 
optical tracking system comprised of two or more fixed detectors such as, for example, fixed 
cameras, one or more energy sources such as, for example, a light source, a passive beacon that 
is reactive to energy from the energy source, and a computer. The computer determines the 
25 location of the passive beacon from the information received from the fixed detectors as the 
detectors receive reflected or transmitted energy from the passive beacon. 

[0016] Yet another aspect of the present invention involves an item tracking system 
comprised of an information gathering device such as, for example, an image device to capture 
data from an object, a beacon detection device to capture information about the orientation and 
30 position of a wearer, a tracking system to follow a passive beacon applied to each object, and a 
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transparent heads-up display showing information related to the object, each in communication 
with one or more computers. 

[0017] One aspect of the invention includes systems and methods for the use of tracking 
technology such as, for example, optical tracking technology, to follow the progress of an object 
5 moving through a complex facility in real time such as, for example, the optical tracking of 
parcels or parts on an assembly line or through a warehouse, 

[0018] Another aspect of the invention includes systems and methods for the use of a 
transparent heads-up display to convey instructions or information to a person when looking at a 
certain object. Such instructions could be for package handling, baggage handling, parts 
10 assembly, navigation through marked waypoints, item retrieval and packaging, inventory control, 
and the like. 

[0019] Yet another aspect of the invention is systems and methods for calibrating an optical 
tracking system using fixed cameras and passive beacons. 

[0020] Another aspect of the present invention provides a system for processing items. The 
15 system is comprised of a tracking system that is configured to provide location information for 
each of a plurality of items on a surface and a display device. The display device is for viewing 
characteristic information for each of the plurality of items at their respective locations. In one 
embodiment, the characteristic information is positioned to indicate the relative position of the 
item on the surface, including putting the characteristic information substantially proximate to a 
20 representation of the item. In another embodiment, only certain characteristic information such 
as, for example, a zip code of a package, is displayed instead of the package at the package's 
position. Items may be singulated or non-singulated. 

[0021] These and other aspects of the various embodiments of the invention are disclosed 
more fully herein. 

25 
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BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S) 
[0022] Having thus described the invention in general terms, reference will now be made to 
the accompanying drawings, which are not necessarily drawn to scale, and wherein: 
[0023] FIG. 1 is an exemplary block diagram of an embodiment of the system of the 
5 invention; 

[0024] FIG. 2 is an embodiment of a data acquisition and display device; 

[0025] FIG. 3 is an embodiment of an exemplary data acquisition and display device as 

shown on a wearer; 

[0026] FIG. 4 is an exemplary diagram of the use of fixed detectors such as, for example, 
10 fixed cameras for a passive beacon location tracking application in an embodiment of the 
invention; 

[0027] FIG. 5 A is an exemplary diagram of the use of fixed detectors such as, for example, 
fixed cameras in a passive beacon location tracking application in an embodiment of the 
invention, and having more detail than the embodiment shown in FIG. 4; 

15 [0028] FIG. 5B is an exemplary view of an image captured by a fixed camera in a passive 
beacon location tracking application, without a filter, in an embodiment of the invention; 
[0029] FIG. 5C is an exemplary view of an image captured by a fixed camera in a passive 
beacon location tracking application, with a filter, in an embodiment of the invention; 
[0030] FIG. 6 is an exemplary illustration of the use of active beacons for determining the 

20 position and orientation of a wearer of a data acquisition and display device in an embodiment of 
the invention; 

[0031] FIG. 7 is an exemplary illustration of the use of passive beacons in an embodiment of 
the invention, as such passive beacons are used for the tracking of items; 

[0032] FIGS. 8A, 8B and 8C are exemplary illustrations of the concept of passive beacon 
25 tracking in an embodiment of the invention; 

[0033] FIG. 9 is an exemplary illustration of a person obtaining an item and placing a retro- 
reflective dot (i.e., a passive beacon) on the item, however, in FIG. 9, the passive beacon is not 
visible as it is underneath the person's thumb; 

[0034] FIG. 10 is an exemplary illustration of a person covering and exposing a passive 
30 beacon with their thumb and causing a "wink"; 



ATL01/11581888vl 



-5- 



AttyDktNo: 018360/253342 



[0035] FIGS. 11 and 12 are exemplary illustrations of the concept of acquiring item 
information (e.g., label information) in an embodiment of the invention; 

[0036] FIG. 13 is a flowchart describing the steps involved in calibrating a fixed camera by 
establishing the fixed camera's position and orientation; 
5 [0037] FIG. 14 is an embodiment of an item tracking system of the invention and is an 
exemplary illustration of the interfaces of such an embodiment; 

[0038] FIG. 15 shows an exemplary application of an embodiment of the system of the 
invention in a parcel sorting facility; 

[0039] FIG. 16 shows an Acquirer aiming a target that is displayed in the see-through display 
10 of the data acquisition and display device at an item's label and placing an adhesive passive 

beacon near the label to trigger the capture of the label image by an image camera; 

[0040] FIG. 17 shows a high-contrast copy of the captured image that is displayed in the 

Acquirer's see-through display so if the captured image appears fuzzy, distorted, or otherwise 

unclear, the Acquirer may re-capture the image; 
15 [0041] FIG. 18 shows exemplary parcels on a conveyer that have come within the Sorter's 

field of view and exemplary superimposed handling instructions proximately on or about parcels 

that are allocated to that Sorter in an embodiment of the invention; 

[0042] FIG. 19 is a flowchart describing the steps for a method of processing an item in an 
embodiment of the invention; 
20 [0043] FIG. 20 also is a flowchart describing the steps for a method of processing an item in 
another embodiment of the invention; 

[0044] FIG. 21 is a flowchart describing a method of displaying information about one or 
more items in a see-through display of a data acquisition and display device in an embodiment of 
the invention; 

25 [0045] FIG. 22 is a flowchart that describes a method of displaying information in a see- 
through display of a data acquisition and display device in another embodiment of the invention; 
[0046] FIG. 23 is a flowchart describing a method of tracking one or more items in an 
embodiment of the invention; 

[0047] FIG. 24 is a flowchart describing a method of tracking one or more items in another 
30 embodiment of the invention; 
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[0048] FIG. 25 is a flowchart describing a method of tracking items in an embodiment of the 
invention; and 

[0049] FIG. 26 is a flowchart that describes a method of computing the orientation and 
position of a wearer of a data acquisition and display device in an embodiment of the invention. 
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DETAILED DESCRIPTION OF THE INVENTION 
[0050] The present invention now will be described more fully hereinafter with reference to 
the accompanying drawings, in which some, but not all embodiments of the invention are shown. 
Indeed, this invention may be embodied in many different forms and should not be construed as 
5 limited to the embodiments set forth herein; rather, these embodiments are provided so that this 
disclosure will satisfy applicable legal requirements. Like numbers refer to like elements 
throughout. 

[0051] The embodiments of the present invention may be described below with reference to 
block diagrams and flowchart illustrations of methods, apparatuses (i.e., systems) and computer 

10 program products according to an embodiment of the invention. It will be understood that each 
block of the block diagrams and flowchart illustrations, and combinations of blocks in the block 
diagrams and flowchart illustrations, respectively, can be implemented by computer program 
instructions. These computer program instructions may be loaded onto a general purpose 
computer, special purpose computer, or other programmable data processing apparatus to 

15 produce a machine, such that the instructions that execute on the computer or other 
programmable data processing apparatus create means for implementing the functions specified 
in the flowchart block or blocks. 

[0052] These computer program instructions may also be stored in a computer-readable 
memory that can direct a computer or other programmable data processing apparatus to function 

20 in a particular manner, such that the instructions stored in the computer-readable memory 
produce an article of manufacture including instruction means that implement the function 
specified in the flowchart block or blocks. The computer program instructions may also be 
loaded onto a computer or other programmable data processing apparatus to cause a series of 
operational steps to be performed on the computer or other programmable apparatus to produce a 

25 computer implemented process such that the instructions that execute on the computer or other 
programmable apparatus provide steps for implementing the functions specified in the flowchart 
block or blocks. 

[0053] Accordingly, blocks of the block diagrams and flowchart illustrations support 
combinations of means for performing the specified functions, combinations of steps for 
30 performing the specified functions and program instruction means for performing the specified 
functions. It will also be understood that each block of the block diagrams and flowchart 
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illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be 
implemented by special purpose hardware-based computer systems that perform the specified 
functions or steps, or combinations of special purpose hardware and computer instructions. 
[0054] Generally, the concepts of the various embodiments of the invention relate to systems 
5 and methods for the processing of singulated and non-singulated items. The embodiments of the 
systems and methods generally involve two sub-systems, a data acquisition and display system 
and a tracking system such as, for example, an optical tracking system. In one embodiment the 
data acquisition and display system includes a set of goggles that have one or more information 
gathering devices such as, for example, cameras, radio-frequency identification (RFED) readers, 

10 barcode readers, RF receivers, etc., or combinations thereof for data capture and a transparent 
heads-up display for displaying data and tracking items. Items may be singulated or non- 
singulated and they may be stationary or moving. Data capturing and tracking for this 
embodiment is initiated by pointing at least one of the information gathering devices on the 
goggles toward a label or tag on an item and initiating tracking of the item by, for example, 

15 uncovering a passive beacon, such as, for example, a retro-reflective dot proximately located on 
each item. The data captured by the goggle's image gathering device is transmitted via a 
network to a local computer that records item data and determines the instructions to be 
displayed in the heads-up display. The local computer may interface with one or more servers 
and business applications. 

20 [0055] In other embodiments, the data acquisition and display may be performed by more 
than one device. For instance, information gathering devices may be mounted on the goggles, or 
they may be separate from the goggles such as wand-mounted or fixed barcode readers, RFID 
readers, cameras, etc. Furthermore, in some embodiments, the display may be separate from the 
goggles, as it may be a fixed display monitor or panel as are known in the art, or it may be a 

25 display affixed to a person by means other than goggle. The display may be of the sort that items 
are viewed through the display and characteristic information about the items is displayed on or 
substantially proximate to the viewed items. In other instances, a representation of one or more 
items may be displayed on the display and characteristic information about the one or more items 
displayed on or proximate to the representations. Furthermore, the characteristic information 

30 may, in some instances, serve as the representation of the item. For example, in a package- 
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handling application, the zip-code of the packages may serve as the representation of the item, 
while also serving as characteristic information about the item. 

[0056] One embodiment of the tracking system is an optical tracking system that includes an 
array of fixed cameras, which track the passive beacons through a sorting and loading facility 
5 and a passive beacon location tracking (PBLT) computer. When a user looks toward a package 
through the goggles, one of the goggle's information gathering devices or a sensor device such as 
a beacon detection device picks up at least two of the active beacon beams. By picking up these 
beams, the local computer is able to determine the location of the user and the user's position. 
The optical tracking system is able to track the location of the uniquely-identified passive 

10 beacons and associate information with each passive beacon. The PBLT computer sends the 
information back to the goggle's local computer via a network, such as for example, a wireless 
network. Therefore, items in the wearer's field of view will have their information appear on the 
heads-up display and will generally appear to be superimposed proximately about or over the 
real objects in the wearer's field of view. Such superimposed information may be applied to the 

15 items in a sequential or random fashion, or it may be applied to all items in the wearer's field of 
view or work area. In one embodiment, only information relevant to that particular wearer will 
be superimposed on the items. Items may be singulated or non-singulated in the wearer's field 
of view. 

[0057] Other embodiments of the tracking system may involve the use of transponders such 
20 as, for example, RFID tags that are attached to or associated with items to be tracked and where 
the location of such transponders is monitored by fixed detectors, as may be known in the art. 
For instance, United States patent number 6,661,335, issued on December 9, 2003 to Seal, fully 
incorporated herein and made a part hereof, describes a system and method for determining the 
position of a RFID transponder with respect to a sensor. 
25 [0058] One embodiment of a data acquisition and display system of the invention is 
comprised of a set of goggles having a see-through display. The term "goggles" is used 
generically and is meant to include any form of lenses (prescription or otherwise), shield or 
shields or even empty frames or other head or body-mounted apparatus capable of having a see- 
through display and one or more information gathering devices or sensors attached thereto. The 
30 see-through display is capable of displaying text and/or images without completely obstructing a 
wearer's line of sight. It may be supported on the head or other part of the body, or in the 



ATL01/11581888vl 



AttyDktNo: 018360/253342 



alternative on a structure that allows a user to view a field of view through the display. The data 
acquisition and display system in some embodiments is comprised of one or more information 
gathering devices such as, for example, cameras that comprise an image-capture camera for 
acquiring label images and a beacon detection device that is used to acquire signals from active 
5 beacons and track orientation and that are attached to the goggles. In other embodiments, the 
label images are acquired by other means such as a fixed image acquisition station located over 
or adjacent to a conveyor belt. The goggles, in some embodiments, may include one or more 
orientation sensors that are used to track a wearer's orientation during times of rapid head 
movement. 

10 [0059] The see-through display, information gathering devices and orientation sensor(s) (if 
included) communicate with a local computer via a network that may be wired, wireless, optical 
or a combination thereof. The local computer may communicate with one or more other 
computers and/or servers over a network and via a network interface. This network may also be 
wired, wireless, optical or a combination thereof. 

15 [0060] In other embodiments, the information gathering devices may be RFED readers, 
barcode readers, RF receivers or transceivers, or combinations thereof. 

[0061] The tracking system includes active beacons that provide a reckoning reference for 
the system to determine position and orientation of wearers of the data acquisition and display 
system and passive beacons that are attached to or associated with each item of interest to 

20 provide a "registration" trigger for each item and to reduce the complexity of the task of three- 
dimensional tracking. The tracking system further includes fixed detectors such as, for example, 
fixed cameras that are used to track an item associated with a passive beacon. An energy source 
such as, for example, a light source is attached to each fixed detector and energy is reflected back 
or returned to the fixed detector by the passive beacons so that the fixed detectors will eliminate 

25 all items except those associated with the passive beacons. In one embodiment the fixed detector 
is a fixed camera and the energy source is a light. A filter on each fixed camera passes reflected 
light from passive beacons such that it provides an image that only shows the passive beacons 
associated with each item of interest. 

[0062] The tracking system provides information to a server or other processor that 
30 communicates with the local computer via a network and may provide information and 
instructions to, or receive information and instructions from, one or more business applications. 
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[0063] FIG. 1 is a block diagram of an embodiment of the system 100 of the invention. This 
embodiment is comprised of a wearable data acquisition and display device 102 combined with 
an optical tracking system 104. The optical tracking system 104 has the ability to track items 
that are associated with passive beacons 128 as such items move throughout a facility. 
5 [0064] Components of the data acquisition and display device 102 are adapted to attach to a 
set of frames, lenses, shields, goggles, etc. (hereinafter generically referred to as "goggles") 106, 
which provides the ability to superimpose information about items that are being tracked 
proximately about or over the real objects (i.e., tracked items) that are within the goggle wearer's 
field of view. This is because the optical tracking system 104 tracks positional information about 

10 items or objects that have passive beacons 128 associated with such items. This tracking occurs 
through the use of fixed cameras 108 and a PBLT computer 110. The item tracking information 
is provided to the data acquisition and display device 102. The data acquisition and display 
device 102 has a local computer 1 12 that calculates the wearer's position and orientation. This is 
accomplished through the use of active beacons 1 14 that have known, fixed locations and unique 

15 "signatures" and a beacon detection device 116 such as, for example, a beacon camera and 
inertial sensor that comprise components of the data acquisition and display device 102. The 
local computer 112 knows the location of the fixed active beacons 114 and from the active 
beacons 114 that are in the beacon detection device's 116 field of view (FOV) is able to 
determine a wearer's position and orientation. Information about tracked items is provided to the 

20 local computer 112 from the optical tracking system 104 via one or more networks 120 and 
network interfaces 122. Therefore, certain information about tracked items that are in the 
wearer's field of view can be displayed on a see-through display 118. This information may 
appear to be superimposed proximately about or on the actual item because of the see-through 
feature of the display 118. 

25 [0065] The information displayed on the see-through display 118 about the tracked item is 
determined by business applications 124 that interface with both, the data acquisition and display 
device 102 and the optical tracking system 104 via the networks 120. For example, these 
business applications 124 may cause sorting and loading instructions to appear on the items so 
that wearer's of the data acquisition and display device 102 do not have to read each item's label 

30 or have to read instructions provided by nearby screens, panels, CRTs, etc. Information about 
the tracked items may be obtained by an information gathering device 126 such as, for example, 
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an image camera that obtains an image of the item's label and registers the item for tracking by 
the optical tracking system 104. The label image may be provided to the local computer 112 
from the image device 126, where it is decoded and provided to the business applications 124 via 
the networks 120. The business applications 124 may combine the label data with other 
5 information and indicate to the local computer 112 what information is to be displayed in the 
see-through display 118. 

[0066] In other embodiments, the information about the tracked items may be obtained by an 
information gathering device 126 such as, for example, a radio frequency identification (RFID) 
reader. In one embodiment, the item's label may be an RFID tag. As previously described, the 

10 information gathering device 126 obtains information from an item's label and registers the item 
for tracking by the optical tracking system 104. The label information may be provided to the 
local computer 112 from the information gathering device 126, where it is decoded and provided 
to the business applications 124 via the networks 120. The business applications 124 may 
combine the label data with other information and indicate to the local computer 112 what 

1 5 information is to be displayed in the see-through display 118. 

[0067] In other embodiments, other tracking systems may be utilized. For instance, a 
tracking system that tracks RFID tags by the use of fixed RFID readers may be used in place of 
an optical tracking system. 
[0068] Data Acquisition and Display Device 

20 [0069] FIG. 2 shows an embodiment of an exemplary data acquisition and display device 
200. The embodiment of the data acquisition and display device 200 shown in FIG. 2 is 
comprised of five components, a set of frames or goggle 202, a see-through display 204, an 
information gathering device such as an image camera 206, a beacon detection device and 
orientation sensor 208, and a local computer 210 having a network interface (not shown). The 

25 see-through display 204 may be, for example, the MicroOptic SV-3 VIEWER TM as is available 
from The MicroOptical Corporation of Westwood, Massachusetts, or similar devices as are 
available from Tek Gear, Inc. of Winnipeg, Manitoba, Kaiser, or Electro-Optics, Inc. of 
Carlsbad, California, among others. The see-through display 204 is used to display 
superimposed objects in the line-of-sight of real objects. The see-through display 204 should 

30 have a resolution sufficient to view the superimposed objects without causing excessive eye 
fatigue. In one embodiment, the resolution of the see-through display 204 may be, for example, 
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a pixel format of 640 columns x 480 rows and have a FOV of at least 75 degrees. The see- 
through display 204 may be either monochrome or color. 

[0070] In other embodiments, the display may be a device separate from the goggle through 
which the items may be viewed or, in other embodiments, on which a representation of the item 
5 may be viewed wherein such representation may include outline images of the items, symbols 
that represents the items or characteristic information about the items. 

[0071] In one embodiment, the beacon detection device 208 is a camera attached to the 
goggles 202 and is used to acquire active beacons 114 (for determining the position and 
orientation of a wearer), and to acquire passive beacons that are in the wearer's field of view. In 
10 one embodiment, the beacon detection device 208 is a beacon camera that is comprised of a 
wide-view (approximately 90° FOV) narrow band camera and orientation sensor. The beacon 
detection device 208 is used to acquire beacons (both active and passive) and the orientation 
sensor is used to track the orientation of the wearer. 

[0072] In the embodiment shown in FIG. 2, the information gathering device is an image 
15 camera 206 that is mounted on the goggle 202. The image camera 206, in one embodiment, is a 
center-view visible light camera that is used to acquire label images. The center-view visible 
light camera (a/k/a the image camera) 206 is used to acquire images and facilitate the registration 
of these images with a passive beacon. In other embodiments, the image camera 206 may be 
separate from the goggle 202. Generally, the image camera 206 will have a depth of field that is 
20 fixed at about 12 inches to 30 inches and a FOV of about 28 degrees. The resolution of the 
image camera 206 in one embodiment is about 1500 x 1500 (2.25 million pixels). An image 
frame capture sequence for the image camera 206 is triggered by the discovery of a passive 
beacon in a close-proximity target zone. The image camera 206 may capture up to 1000 images 
per hour. 

25 [0073] The goggles 202 should provide the wearer with a sufficient FOV such that the 
wearer does not have to continuously move their head back and forth. In one embodiment, this 
FOV is provided by goggles 202 having at least a 75 degree FOV, although other degrees of 
FOV may be used. 

[0074] The local computer 210 is comprised of a computer and network interface (not 
30 shown) that determine the orientation and position determination of the wearer from images 
obtained from the beacon detection device and orientation sensors 208. The local computer 210 
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also performs view-plane computations, which is a process that uses the three-dimensional 
position data for each relevant object, and determines the position and orientation of the wearer 
of the data acquisition and display device 200. The local computer 210 manages the application- 
provided display symbology for each relevant object to determine what is to be displayed in the 
5 see-through display 204 and where to display the information such that it appears superimposed 
proximately about or on the item. The local computer 210 performs close-proximity passive 
beacon discovery and registration, information processing such as image capture from the image 
capture camera 206, calibration of the beacon detection device 208 and image camera 206 with 
the see-through display 204, calibration of active beacons 114 relative to fixed cameras 108, 

10 communications (generally, wireless), and machine-readable codes decoding, which is a 
capability that significantly reduces the response time for displaying information on already- 
registered objects. For example, the system 100 has ready to display information on an object 
and the object becomes obscured for a while and then re-appears; the user re-registers the object 
and quickly sees the relevant information; on-board decoding avoids the time to transfer the 

15 image across the communications network 120 to the business applications 124 for determination 
of display information. In one embodiment, for example, the local computer 210 may be a 250 
MHz low power consumption CPU. 

[0075] The local computer 210 packaging may also contain a power source (not shown), 
which may be self-contained such as, for example, batteries or other forms of rechargeable, 
20 replaceable, reusable or renewable power sources. In one embodiment, for example, the power 
source is 10-volt, 3 amp-hour battery. 

[0076] In the embodiment of FIG. 3, the local computer 210 communicates with the goggle- 
mounted devices 204, 206, 208 via a cable 212. In other embodiments, however, such 
communication may occur wirelessly, through fiber optics, or combinations thereof. FIG. 3 is an 
25 embodiment of the data acquisition and display device 302 as shown on a wearer 304. As shown 
in the embodiment of FIG. 3, the data acquisition and display device 302 is comprised of a see- 
through display 306 that is attached to or incorporated into a set of frames or goggles 308, and 
one or more information gathering devices such as cameras, and orientation sensors 310 attached 
to the frames 308. 

30 [0077] The frames 308 are head-mounted on a wearer 304, similar to a pair of glasses or 
goggles. A local computer 312 communicates with the see-through display 306, information 
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gathering devices, and orientation sensors 310, optical tracking system 104, and business 
applications 124 over one or more networks. 
[0078] Tracking Systems 

[0079] FIG. 4 is an exemplary diagram of the use of fixed detectors fixed cameras in a 
5 passive beacon location tracking application in an embodiment of the invention. The fixed 
detectors such as, for example, fixed cameras 402 are mounted at fixed positions in the vicinity 
of the objects of interest 404. The purpose of these fixed cameras 402 is to continuously provide 
images to the process that computes the current location of each object of interest (a/k/a "items") 
404. The objects of interest 404 may be singulated (as shown), or non-singulated. Each object 

10 of interest 404 is associated with at least one passive beacon 406. 

[0080] FIG. 5C is an exemplary diagram of the use of fixed detectors such as, for example, 
fixed cameras 504 in a passive beacon location tracking application in an embodiment of the 
invention and having more detail than FIG. 4. In this embodiment, an energy source such as, for 
example, a light source 502 is attached to each fixed camera 504 and aimed along the image path 

15 506. The light source 502 is generally not visible to the human eye (e.g., infrared), although in 
other embodiments other visible or non-visible light sources may be used such as, for example, 
lasers, colors or colored lights, ultraviolet light, etc. The lens 508 of the camera 504, in one 
embodiment as shown in FIG. 5C, is covered with a filter 510 that is matched to the frequency of 
the light source 502. The purpose of the light source 502 and filter 510 is to provide an image 

20 512 that only shows passive beacons 514 that are attached to or associated with each singulated 
or non-singulated item of interest 516, as shown by the images 512, 518 of FIGS. 5C and 5B, 
respectively. In one embodiment, the fixed cameras 504 are low-cost, web-cam type cameras 
having a resolution of about 640 x 480 pixels. 

[0081] FIG. 6 is an exemplary illustration of the use of active beacons 602 for determining 
25 the position and orientation of a wearer 304 of a data acquisition and display device 102 in an 
embodiment of the invention. The active beacons 602 provide a reckoning reference for the 
local computer 112 to determine the position and orientation of a user wearing the device 102. 
In one embodiment, the active beacons 602 are sources of blinking light that are each uniquely 
recognized by the beacon detection device 1 16 of the data acquisition and display device 102. In 
30 other embodiments, the active beacon 602 may be any source of unique magnetic, electrical, 
electronic, acoustical, optical transmission that are recognizable by the beacon detection device 

ATL01/11581888vl -16- AttyDktNo: 018360/253342 



1 16 of the data acquisition and display device 102. Each active beacon 602 has a relative fixed 
position 604 such as, for example, three-dimensional coordinates x, y, and z. The relative fixed 
position 604 of each active beacon 602 is known to the local computer 112, therefore the relative 
position and orientation of a wearer of the data acquisition and display device 102 may be 
5 computed by the local computer 1 12 by determining which active beacons 602 are in the FOV of 
the beacon detection device 116 of the data acquisition and display device 102. 
[0082] Generally, the energy source of the active beacon 602 is infrared light, although other 
visible or non-visible sources may be used such as lasers, colors or colored lights, ultraviolet 
light, etc. Furthermore, in some instance, each active beacon 602 may use unique non-optical 

10 signals such as, for example, electronic transmissions, acoustical, magnetic, or other means of 
providing a unique signal for determining the orientation and position of the wearer 304. 
[0083] In an embodiment where the active beacon 602 is a source of blinking infrared light 
and the beacon detection device 116 is a beacon camera, each active beacon 602 is uniquely 
identified by a blinking pattern that differentiates each active beacon 602 from other light 

15 sources and from other active beacons. For example, in one embodiment each active beacon 602 
transmits a repeating 11 -bit unique identification pattern. This pattern consists of a 3-bit 
preamble followed by an 8-bit ID value. For instance, the preamble may be "001" and the ID 
value may be one of 88 values that do not begin with or contain the string "001." Each pattern 
bit is split into two transmit bits. The state of the transmit bit determines whether the beacon is 

20 on or off. The value of the transmit bits are determined using a standard technique called 
"alternate mark inversion" or AMI. AMI is used to ensure that the beacon has a reliable blink 
rate. AMI is generally encoded whereby a "0" information bit becomes "01" and a "1" 
information bit alternates between "11" and "00." The duration of the transmit bit is a little 
longer than the frame capture interval of the beacon camera 116. This is so that the beacon 

25 camera 116 does not miss any blink states. Assuming, for example, a 10 frames per second 
frame rate, the transmit bit will last for about 110 milliseconds. Therefore, the time for the active 
beacon to cycle through the entire identification cycle is: 11 bits x 2 transmit bits x 110 
milliseconds = 2.4 seconds. The on/off cycle of each active beacon 602 is about 220 
milliseconds or 440 milliseconds. The beacon detection device 116 of this embodiment is able to 

30 isolate beacon 602 blinkers from background noise by filtering out all light sources that do not 
have the given frequency. 
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[0084] FIG. 7 is an exemplary illustration of the use of passive beacons 702 in an 
embodiment of the invention, as such passive beacons 702 are used for the tracking of items 704. 
The passive beacon 702 is intended to be a low-cost item that is attached to or associated with 
each item of interest 704. Its purpose is to provide a registration trigger for each item 704 and to 
5 provide a reference point to aid in three-dimensional position tracking from image data, as 
obtained from the fixed cameras 504. In one embodiment, the passive beacon 702 is a use-once, 
adhesive light reflector, such as retro-reflective dots available from 3M of St. Paul, Minnesota. 
Retro-reflection causes light from a certain location to be reflected back, without extensive 
scattering, to the source of the light. The light source 502 attached to each fixed camera 504 

10 (previously described - see FIG. 5 A) is reflected back to the fixed camera 504. Because most 
other extraneous sources of light (noise) will be from sources less-reflective than the retro- 
reflective dots, the image viewed by the fixed camera 504 will be easily processed to eliminate 
most shapes except for the passive beacons 702. Generally, a passive beacon 702 having a 
diameter of approximately one-half inch will provide the resolution necessary for the fixed 

1 5 cameras 504 at a reasonable range. 

[0085] In other embodiments, the passive beacon may be an RFK) tag located on or 
associated with the item. A modulated RFK) signal is returned from the RFID tag passive 
beacon when a certain RF signal is present. Further, such a passive beacon overcomes 
challenges associated with passive beacons that must maintain a certain orientation toward a 

20 detector. For instance, an RFK) passive beacon could continue to be tracked if the item is 
flipped over or if it passes under some obstructions. As previously described, United States 
patent number 6,661,335, incorporated fully herein, describes a system and method for tracking a 
RFK) transponder relative to a sensor (e.g., fixed detector). 

[0086] The process involved in the optical tracking system knowing the position of the 
25 passive beacons 702 is two-part; passive beacon registration and passive beacon tracking. 

[0087] The concept of passive beacon tracking is illustrated in the embodiment shown in 
FIGS. 8A, 8B and 8C. Passive beacon tracking occurs once a passive beacon 806 has been 
detected by two or more fixed detectors such as, for example, fixed cameras 804, 804a. The 
three-dimensional computed position 802 of the passive beacon 806 is determined from knowing 
30 the position and orientation of each fixed camera 804, 804a. The passive beacon location 
tracking system 110 computes the passive beacon's position from two-dimensional images 
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(FIGS. 8B and 8C) from the fixed cameras 804, 804a that are interpolated to be synchronized in 
time that track the position of passive beacon 806 relative to the location 808, 808a of each of the 
fixed cameras 804, 804a. 

[0088] The passive beacon location tracking system 110 should keep track of a passive 
5 beacon 802 during periods of intermittent disappearance and when the passive beacons 802 are 
visible to only one fixed camera 804 to provide consistent tracking. Two fixed cameras 804 first 
acquire a passive beacon 802 to initially determine the passive beacon's location, but a "lock" is 
maintained while the passive beacon 802 is visible to only one fixed camera 804. The passive 
beacon location tracking system 110 makes assumptions about the passive beacon's motion that 

10 enable the lock to be maintained during times of disappearance. For example, streams of passive 
beacons associated with items flowing along on a conveyor system (as shown in FIGS. 5A and 
5C) have a high likelihood of not flowing backward. The probable trajectory of the passive 
beacon 802 is used by an algorithm of the passive beacon location tracking system 110 to track 
the unobserved passive beacon 802. It may also be possible to track passive beacons 802 

15 flowing under a conveyor over-pass by observing continuous flow. However, when a passive 
beacon 802 falls out of view of all fixed cameras 804 for a significant period of time, the passive 
beacon location tracking system 110 loses the item and it (the passive beacon 802) is essentially 
gone from the perspective of the passive beacon location tracking system 110. 
[0089] FIGS. 9 and 10 provide exemplary illustrations of the concept of passive beacon 

20 registration, in an embodiment of the invention. Passive beacon registration occurs when a 
passive beacon is being detected simultaneously by two or more fixed detectors and the passive 
beacon location tracking system 110 declares that the passive beacon is discovered. In an 
embodiment having a passive beacon comprised of reflective material and fixed detectors 
comprised of fixed cameras, the passive beacon location tracking system discovers a passive 

25 beacon when a prominent reflection (generally, an infrared reflection) "winks" at the beacon 
detection device 116 (in this instance, a beacon camera). In FIG. 9, a person wearing a data 
acquisition and display device 102 has obtained an item 902 and has placed a retro-reflective dot 
(i.e., a passive beacon) 904 on the item 902. In the embodiment of FIG. 9, the passive beacon 
904 is not visible as it is underneath the person's thumb. In FIG. 10, the person has moved their 

30 thumb, thereby exposing the passive beacon 904, and causing a "wink." The "wink" is a sudden 
long-duration (greater than approximately one-half second) steady reflection from the passive 
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beacon 904. The "wink" is also observed by the fixed cameras 108 of the optical tracking 
system 110. The local computer 112 of the data acquisition and display device 102 assigns the 
newly-acquired passive beacon 904 a unique handle. The data acquisition and display device 
102 notifies the passive beacon location tracking system 1 10 of the passive beacon 904 discovery 
5 and its handle, as well as the approximate location of the discovered passive beacon 904. 

[0090] The passive beacon location tracking system 110 relates the discovered passive 
beacon's handle to the tracked passive beacon that was observed to "wink" at the fixed cameras 
108. The optical tracking system 104 acknowledges the lock-on of the passive beacon 904 to the 
data acquisition and display device 102, allowing the data acquisition and display device 102 to 

10 provide positive feedback of tracking to the wearer. The optical tracking system 110 publishes, 
and continually updates, the three-dimensional position of the passive beacon 904 relative to the 
passive beacon's 904 given unique handle. In other embodiments, the "winking" process may be 
performed by mechanical shutters between the passive beacon and the fixed cameras 108 and/or 
image device 206, by adjusting the apertures of the cameras 108, 206, or by "self-winking" or 

1 5 blinking passive beacons 904. 

[0091] FIGS. 11 and 12 illustrate the concept of acquiring item information (e.g., label 
information) in an embodiment of the invention. In this embodiment, the information gathering 
device is an image camera 206. The image camera 206 of this embodiment of the data 
acquisition and display system 200 acquires the image 1102 from the item 1104. The local 

20 computer 210 of the data acquisition and display device 200 receives the image 1102 from the 
image camera 206 and decodes machine-readable codes (e.g., barcodes, etc.) from the image and 
passes the image 1102 and decoded information for the related passive beacon handle to any 
associated business applications 124. These business applications 124 assign relevant 
displayable information that will be presented to designated wearers of a data acquisition and 

25 display device 200 when the passive beacon's 904 three-dimensional position is within the see- 
through display's 204 field of view and within range. In another embodiment (not shown) the 
"label" is an RFID tag and the information gathering device 126 is an RFID reader. In yet other 
embodiments (not shown), the item information may be acquired by fixed devices or devices 
separate from the data acquisition and display device, as such devices are known in the art. In 

30 the particular embodiment of FIG. 1 1, an image of the acquired information 1 102 is displayed on 
or proximate to the item 1 104 to verify acquisition of the information. 
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[0092] Orientation Of The Data Acquisition And Display Device 

[0093] The local computer 112 uses real-time information derived from the beacon detection 
device 116 to determine orientation and position of the data acquisition and display device 102, 
and thus any wearer of the device 102, relative to the active beacons 114. The orientation 
5 information derived from the beacon detection device 116 is augmented by highly responsive 
inertial three degrees-of-freedom (DOF) rotational sensors (not shown separately from 116). 
[0094] The orientation information is comprised of active beacon IDs and active beacon two- 
dimensional image position from the beacon detection device 116. Additional information that is 
needed includes the active beacons' three-dimensional reference locations versus the active 

10 beacons' IDs. Multiple active beacons 1 14 are used to determine the data acquisition and display 
device's 102 orientation and position. The more active beacons 114 used to compute orientation 
and position, the greater the accuracy of the measurement. Also, it may be possible that a 
particular active beacon ID value is used for more than one active beacon in a particular facility. 
Therefore, the data acquisition and display device 102 must be able to discard position values 

15 that are non-determinant (i.e., non-solvable positions from beacon images). 

[0095] Because of the relatively slow nature of the active beacon ID transmission sequence, 
the tracking design must accurately assume the identification of each active beacon 114 for each 
updated image capture frame. Once an active beacon 1 14 is identified, the data acquisition and 
display device 102 must "lock-on"" and track its motion (as caused by movement of the wearer) 

20 in the two-dimensional image plane. The known unique blink or transmission rate, pattern or 
signal of the active beacons 1 14 allows the image processor to remove most energy sources from 
the image that are not active beacons 114 by use of a filter such as, for example, a narrow-pass 
filter. The remaining active beacons are identified after observing a complete ID cycle 
(previously described). The extrapolated two-dimensional position of each identified active 

25 beacon 1 14 is input into the three-dimensional position and orientation computation process. 
[0096] Inertial Navigation 

[0097] Because it may be difficult to track a wearer's head movement with active beacons 
114 when the wearer's head moves relatively quickly, inertial sensors, in combination with the 
beacon detection device 116, may be used in these instances to determine head orientation. 
30 Inertial navigation technology, in one embodiment, uses semiconductor-sized micro-machined 
accelerometers to detect rotation. Such devices are commercially available from manufacturers 
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such as, for example, InterSense, Inc. of Burlington, Massachusetts, among others. The inertial 
navigation sensors may replace or supplement the active beacon 114 orientation signal during 
times of rapid head movement. 

[0098] Calibration (Positioning) of Fixed Detectors 
5 [0099] The process of installing fixed detectors such as, for example, fixed cameras 108 and 
establishing their known position in relation to other fixed cameras 108 is a multi-step process 
whereby multiple fixed cameras 108 observe the same object and learn their position and 
orientation relative to one another. Referring to the flowchart FIG. 13, the following steps are 
involved in establishing a fixed detector's position and orientation: the process begins with Step 

10 1300. In Step 1302, the first and second fixed detectors to be calibrated are chosen because they 
are installed adjacent (with a normal separation distance for tracking) to each other. In Step 
1304, the tracking system 104 is placed into calibration mode for the two fixed detectors of 
interest. In Step 1306, a passive beacon 904 is placed within view of both fixed detectors and the 
passive beacon is covered or blocked and uncovered several times so as to cause a "winking" 

15 effect, thus causing the tracking system 104 to calculate the possible positions and orientations of 
both fixed detectors relative to one another. In Step 1308, the passive beacon 904 is repositioned 
to a different location within view of both fixed detectors and the "winking" procedure of Step 
1306 is repeated. In Step 1308, the passive beacon repositioning/winking process is repeated 
until the tracking system 104 indicates that a single unique position is known for each fixed 

20 detector, which may take between two and four iterations of the repositioning/winking process. 
In Step 1310, the third through the remaining fixed detectors are calibrated in a similar 
repositioning/winking process until all fixed detectors are calibrated. If a fixed detector will not 
calibrate during the repositioning/winking process, it may be installed incorrectly and need to be 
re-installed or repaired. The process ends at Step 1312. When a new fixed detector is installed 

25 or an old fixed detector is moved, the repositioning/winking process is performed so that the 
detector's new position is learned relative to the calibrated adjacent detectors. 
[0100] Calibration of Data Acquisition and Display Device 

[0101] The data acquisition and display device 200 is calibrated so that the alignment 
between the devices of the data acquisition and display device 200 is known. It is assumed that 
30 normal manufacturing tolerances and routine use will result in some amount of mis-alignment of 
the active beacon detection device 208, information gathering device such as an image camera 
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206, and the see-through display 204. These devices require concurrent alignment for better 
operational characteristics of the data acquisition and display device 200. The procedure 
requires first placing the data acquisition and display device 200 into calibration mode by aiming 
the image camera 206 at a special pattern or barcode. A crosshair pattern is then displayed on 
5 the see-through display 204 and the crosshairs are aimed at the special calibration pattern. The 
see-through display 204 will then ask for successive trials of aiming the crosshairs of the see- 
through display 204 until the data acquisition and display device 200 is able to isolate the needed 
precision in the alignment compensation for the imaging camera 206, beacon detection device 
208, and the see-through display 204. This calibration information will be retained by the data 
10 acquisition and display device 200 until the next calibration mode process. 
[0102] Calibration Of Active Beacons 

[0103] The position of each active beacon 114, relative to the fixed detectors such as, for 
example, fixed cameras 108, must be known so that the data acquisition and display device 102 
can determine the position and orientation of a wearer relative to the active beacons 114. The 

15 calibration process begins by attaching an active beacon 114 to the side of each of three 
calibrated and adjacent fixed cameras 108 or by having three active beacons 114 with known 
locations. The positions of these active beacons are now known from the positions of the fixed 
cameras 108. A fourth active beacon 114 is placed anywhere within the field of view of the 
beacon detection device 116 along with the three initially placed active beacons 114 having 

20 known locations. With a calibrated data acquisition and display device 102 that has been placed 
in its active beacon calibration mode, the wearer aims the crosshairs displayed in the see-through 
display 1 18 at the fourth active beacon 114. The wearer is then prompted to reposition the data 
acquisition and display device 102 (while still maintaining the three active beacons 114 with 
known locations and the fourth active beacon 114 in the field of view of the beacon detection 

25 device 116) several times until a location for the fourth active beacon 114 is computed by the 
local computer 112. This process is repeated as active beacons 114 are added throughout the 
facility. Anytime a new or moved active beacon 114 is installed, this aiming and calibration 
process with a data acquisition and display device 102 will determine the relative location of the 
active beacon 114. 

30 [0104] The installer of the active beacon 114 chooses the physical ID values for each active 
beacon 114. The installer should not use equivalent IDs on active beacons 114 that are adjacent 

ATL01/11581888vl -23- AttyDktNo: 018360/253342 



to a common active beacon 114. One way to prevent this is to section the facility off into 
repeating 3x3 grid zones, zones "a" through "i." All active beacons 114 installed in an "a" zone 
are assigned an ID from a pre-determined "a" set of IDs, all active beacons installed in an "b" 
zone are assigned an ID from a pre-determined "b" set of IDs, etc. The size of each zone is a 
5 function of the number of active beacons 1 14 that may be maximally required in each zone. The 
3x3 grid is repeated throughout the facility as often as needed. The random nature of active 
beacon locations generally prevents any two zones within the facility from having the exact 
relative positioning of active beacons 114 within each zone. Each active beacon 114 in an 
installation has a unique logical ID value (previously described) that is assigned to the 
10 combination of a physical ID value and a three-dimensional position. The active beacon 
installation process produces and assigns the logical ID value. 
[0105] Component Interfaces 

[0106] Referring to FIG. 14, the optical tracking system 1402 of this embodiment is designed 
to be as self-contained as possible. A passive beacon location tracking ("PBLT") computer 1404 

15 accepts all fixed camera 1406 images and, with the known relative position and orientation of the 
fixed cameras 1406, uses the images to determine the three-dimensional location of each tracked 
passive beacon 1408. The optical tracking system 1402 is comprised of one or more inputs from 
an information gathering device 1412 of one or more data acquisition and display devices 1410 
that cue the registration of a passive beacon 1408 for tracking; the fixed cameras 1406 from 

20 which the PBLT 1404 reads all images from each fixed camera 1406; a fixed camera locations 
repository 1414 that contains each fixed camera's logical ID, position and orientation and is used 
to calculate the positions of all tracked passive beacons 1408, and is updated when the PBLT 
1404 is in fixed camera installation mode; object location repository 1416, which stores the 
location of each passive beacon (or item) 1408 by the item's logical ID (may be accessed by 

25 business applications); and, a maintenance console (not shown in FIG. 14), which is a user 
interface that provides information about the optical tracking system's 1402 configuration and 
controls the installation mode for the fixed cameras 1406. The passive beacons 1408 are 
generally associated with items (e.g., parcels) 1432, so that the items may be tracked. 
[0107] Application Interfaces 

30 [0108] Still referring to FIG. 14, in addition to providing information to wearers of a data 
acquisition and display device 1410, the optical tracking system 1402 is capable of providing 
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information to other business applications 1418. For example, in one embodiment, the business 
application receives an item's logical ID and decoded label information of the item from the data 
acquisition and display device 1410. The business application 1418 converts the label 
information into display information and publishes the information to a data repository 1420 that 
5 contains object ID information and associated display information. By cross-referencing the 
object ED information with the object location repository 1416 of the optical tracking system 
1402, this information can be provided to a data acquisition and display device 1410 that, by 
knowing its position and orientation as determined by an orientation computation process of the 
local computer 1422, the display information can be displayed on the see-through display 1424 

10 such that it is properly associated with the object. The orientation computation process involves 
accessing an active beacons location database 1426 containing the know locations of active 
beacons 1428 and a unique identifier assigned to each active beacon 1428 such that when a 
wearer of a data acquisition and display device 1410 detects certain active beacons 1428 by their 
assigned identifier with the data acquisition and display device's beacon detection device 1430, 

15 the local computer is able to compute the orientation and position of the data acquisition and 
display device 1410. 

[0109] In another embodiment, the business application 1418 receives images of objects and 
converts the images into display information. In other embodiments, the business application 
1418 receives a logical ID value for the data acquisition and display device 1410 that provided 

20 the information, along with decoded label data. If the decoded label data is of the type that is 
application-defined to represent a job indicator, then the business application 1418 is able to 
discern which data acquisition and display device 1410 is assigned to each job type and display 
information is provided to only this data acquisition and display devices 1410. Finally, the 
business application 1418 receives an item's logical ID along with the item's position from the 

25 optical tracking system 1402. The business application 1418 uses the position information to 
determine the status of certain items, project processing times, measure throughput of items in a 
facility, and make other business decisions. 
[0110] System Operation Example 

[0111] An exemplary method of applying an embodiment of the system of the present 
30 invention is its use in a parcel sorting facility as shown in FIG. 15. In this example, a data 
acquirer ("Acquirer") 1502 and a parcel sorter ("Sorter") 1504 wear and use a data acquisition 
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and display device 200 in the performance of their duties. However, in other embodiments, the 
step of acquiring item information may be performed by devices not connected to a data 
acquisition and display device 200 such as by an over-the-belt scanning system, as are known in 
the art. Others, such as supervisors and exception handlers may also wear a data acquisition and 
5 display device 200, but those persons are not described in this particular example. 

[0112] In a first step, the Acquirer 1502 and Sorter 1504 each don a data acquisition and 
display device 200, power it up, and aim the information gathering device such as, for example, 
an image camera 206 at a special job set-up indicia, pattern, or barcode that is application 
defined. The chosen business application, as selected by the job set-up indicia, is notified by 
10 each data acquisition and display device 200 of the initialization and job set-up. The business 
application thus becomes aware of the data acquisition and display devices 200 that are 
participating in each job area. 

[0113] The Acquirer 1502 is positioned near the parcel container unload area 1506 of the 
facility and images the shipping label of each parcel 1508. As shown in FIG. 16, the Acquirer 

15 1502 aims a target 1602 that is displayed in the see-through display 204 of the data acquisition 
and display device 200 and places a passive beacon such as, for example, an adhesive reflective 
passive beacon 1604 near the label 1606. The passive beacon 1604 is covered and uncovered 
thereby "winking" the passive beacon 1604 at the beacon detection device 208 of the data 
acquisition and display device 200 and triggering the capture of the label image by the image 

20 camera 206. In other embodiments (not shown), label information may be captured by over-the- 
belt label readers or other such devices, as they are known in the art. 

[0114] In a registration step, the optical tracking system 1402 detects the appearance of a 
passive beacon 1604 through the fixed detectors such as, for example, the fixed cameras 108 and 
receives a notification event from a data acquisition and display device 200 that assigns a logical 
25 ID value to the passive beacon 1604. The optical tracking system 1402 begins tracking the 
passive beacon 1604 and sends a track lock-on acknowledgement to the data acquisition and 
display device 200. 

[0115] As shown in FIG. 17, in this embodiment, a high-contrast copy of the captured image 
1704 is displayed in the Acquirer's 1502 see-through display 204 to indicate that the label 
30 information has been captured. If the captured image 1704 appears fuzzy, distorted, or otherwise 
unclear, the Acquirer 1502 may re-capture the image 1704. The see-through display 204 of the 
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data acquisition and display device 200 will also display a confirmation to the Acquirer 1502 that 
the tracking process for the item has begun and that the Acquirer 1502 may move on to the next 
parcel. If the Acquirer 1502 does not receive the confirmation or if the images need to be re- 
captured, then the passive beacon 1604 should once again be "winked" in order to repeat the 
5 acquisition cycle. If confirmation is received and the image does not need to be re-captured, the 
item is placed on a conveyor system 1512 with the passive beacon 1604 facing the fixed cameras 
108. 

[0116] While the acquired parcels 1508 travel in either a singulated or non-singulated 
manner on the conveyor 1512, the business application uses the decoded label data acquired 
10 from the image to determine appropriate handling instructions for each parcel 1508. If the label 
has insufficient coded data, then the image from the label is transferred to a key-entry 
workstation. Using the label image, the key-entry personnel will gather the information needed 
to handle the package. 

[0117] Each Sorter 1504 wearing a data acquisition and display device 200 has a defined 

15 field of view (FOV) 1510, as shown in FIG. 15. Once one or more parcels 1508 on the conveyer 
1512 comes within the Sorter's FOV 1510, as shown in FIG. 18, the Sorter 1504 will see that 
package's 1802 super-imposed handling instructions 1804 proximately floating over or about the 
packages 1802 that are allocated to that Sorter 1504. The Sorter 1504 will load each of these 
packages 1508 according to the super-imposed handling instructions 1804. In one embodiment, 

20 tracked packages 1508 on the conveyor 1512 that have somehow lost their handling instructions 
have a special indicator (not shown) imposed on them and can be re-registered by "winking" 
their passive beacon 1604 thus causing the super-imposed handling instructions 1804 to appear 
to wearers of a data acquisition and display device 200. In some embodiments, tracked packages 
1508 that are not allocated to the immediate area of a Sorter 1504 have a special symbol (not 

25 shown) super-imposed on them. This indicates that the package is being tracked, but that it is 
not for loading in that Sorter's 1504 immediate area. In some embodiments, packages that have 
no handling instructions or special symbol associated with them provides indication that the 
package was never registered by the Acquirer 1502 or that the package has been flipped or 
otherwise lost its passive beacon 1604. In one embodiment, parcel information is displayed 

30 sequentially as each package 1508 enters a Sorter's 1504 field of view 1510 or work area, 
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whereas in other embodiments information is displayed for all parcels 1508 within the Sorter's 
1504 field of view 1510 or work area. The parcels 1508 may be singulated or non-singulated. 
[0118] FIG. 19 is a flowchart describing the steps for a method of processing an item in an 
embodiment of the invention. The steps include beginning the process at Step 1900. At Step 
5 1902, an item is viewed while wearing a data acquisition and display device having a see- 
through display. Step 1904 involves displaying processing instructions on the see- through 
display in a manner such that the processing instructions appear proximately superimposed on 
the item. In Step 1906, the items are processed in accordance with the processing instructions. 
The process ends at Step 1908. Such a process as described in FIG. 19 may be used for the 

10 processing of mail and parcels, among other uses. 

[0119] FIG. 20 is also a flowchart describing the steps for a method of processing an item in 
another embodiment of the invention. The process of FIG. 20 begins at Step 2000. At Step 
2002 an item is tracked with a tracking system as the item's location changes. At Step 2004, the 
orientation and position of a wearer of a data acquisition and display device having a see-through 

15 display is determined. At Step 2006, it is determined which items are in the field of view of the 
see-through display of the data acquisition and display device. In Step 2008, an item is viewed 
through the see-through display of the data acquisition and display device. In Step 2010, 
processing instructions relevant to the item are displayed on the see-through display in a manner 
such that the processing instructions appear proximately superimposed on the item. In Step 

20 2012, the item is processed in accordance with the processing instructions. The process ends at 
Step 2014. 

[0120] FIG. 21 is a flowchart describing a method of displaying information about one or 
more items in a see-through display of a data acquisition and display device in an embodiment of 
the invention. The process begins at Step 2100. At Step 2102, orientation and position 

25 information about a wearer of the data acquisition and display device is captured. At Step 2104, 
a field of view of the see-through display is determined from the captured orientation and 
position information. At Step 2106, information is displayed on the see-through display about 
the items in the field of view of the see-through display such that the information appears to be 
proximately superimposed on the items when the items are viewed through the see-through 

30 display. The process ends at Step 2108. Such a process as described in FIG. 21 may be used for 
the processing of mail and parcels, among other uses. 
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[0121] FIG. 22 is a flowchart that describes a method of displaying information in a see- 
through display of a data acquisition and display device in another embodiment of the invention. 
The process begins at Step 2200. In Step 2202, data about an item is captured by, for example, 
an information gathering device such as the image device 126. In Step 2204, information and 
5 instructions about the item are determined from the captured data. In Step 2206, orientation and 
position information about a wearer of the data acquisition and display device is captured by, for 
example, the beacon detection device 116. In Step 2208, a field of view of the see-through 
display of the data acquisition and display device is determined from the captured orientation and 
position information. In Step 2210, information and instructions are displayed on the see- 
10 through display about the item in the field of view of see-through display such that the 
information and instructions appear to be proximately superimposed on the item when the item is 
viewed through the see-through display. The process ends at Step 2212. 

[0122] FIG. 23 is a flowchart describing a method of optically tracking one or more items in 
an embodiment of the invention. The process begins at Step 2300. At Step 2302, a source of 

15 energy such as, for example, a light, magnetic waves, electronic transmission, etc. is provided. 
In Step 2304, a passive beacon such as, for example, a retro-reflective dot or other shape 
comprised or retro-reflective material is placed on or associated with an item. The passive 
beacon is activated by the source of energy or said beacon reflects energy from the source of 
energy. In Step 2306, two or more fixed detectors such as, for example, fixed cameras having 

20 known fixed locations relative to one another are provided with each fixed camera having a 
defined field of view and capable of detecting energy transmitted or reflected from the passive 
beacon if the passive beacon is in the fixed camera's field of view. In Step 2308, the location of 
the passive beacon is computed from the energy received by the two or more fixed cameras from 
the passive beacon as the location of the item changes. The process ends at Step 2310. The 

25 process as described above may be used for the optical tracking of mail and parcels, among other 
uses. 

[0123] FIG. 24 is a flowchart describing a method of optically tracking one or more items in 
another embodiment of the invention. The process begins at Step 2400. At Step 2402, a source 
of energy such as, for example, a light, magnetic waves, electronic transmission, etc. is provided. 
30 In Step 2404, a passive beacon such as, for example, a retro-reflective dot or other shape 
comprised or retro-reflective material is placed on an item. The passive beacon is activated by 
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the source of energy or said beacon reflects energy from the source of energy. In Step 2406, two 
or more fixed detectors such as, for example, fixed cameras having known fixed locations 
relative to one another are provided with each fixed camera having a defined field of view and 
capable of detecting energy transmitted or reflected from the passive beacon if the passive 
5 beacon is in the fixed camera's field of view. In Step 2408, the location of the passive beacon is 
computed from the energy received by the two or more fixed cameras from the passive beacon as 
the location of the item changes. In Step 2410, a data acquisition and display device having a 
see-through display, an image device such as, for example, an image camera or an RFID reader, 
a local computer, and a beacon detection device such as, for example, a beacon camera, is 

10 provided. In Step 2412 image data about the item is captured with the image device. The image 
data may be, for example, a mailing label having both machine-readable and human-readable 
elements, or an RFID tag, or a combination thereof. In Step 2414, information about the item is 
determined from the image data with the local computer. In Step 2416, orientation and position 
information about the data acquisition and display device is captured with the beacon detection 

15 device. In Step 2418, a field of view of the see-through display is determined from the captured 
orientation and position information. In Step 2420, it is determined if the item is in the field of 
view of the see-through display from the location of the passive beacon. In Step 2422, 
information and instructions are displayed on the see-through display about the item if the item is 
in the field of view of see-through display such that the information and instructions appear to be 

20 proximately superimposed on the item when the item is viewed through the see-through display. 
The process ends at Step 2424. 

[0124] FIG. 25 is a flowchart describing a method of tracking items in an embodiment of the 
invention. The process begins with Step 2500. In Step 2502, a data acquisition and display 
device having an information gathering device to capture data about an item is provided. The 

25 information gathering device may be, for example, an image camera, an RFID reader, etc. The 
captured data may come from a mailing label and/or an RFID tag. Also provided is an active 
beacon detection device to capture orientation and position information about a wearer of the 
data acquisition and display device, a see-through display to display information and instructions 
about the item, and a local computer in communication with the information gathering device, 

30 active beacon detection device, and see-through display. The local computer decodes data from 
the information gathering device, computes the orientation and position of the wearer of the data 
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acquisition and display device from the orientation and position information captured by the 
active beacon detection device, and provides information and instructions to be displayed in the 
see-through display about items in the field of view of the data acquisition and display device. 
[0125] In Step 2504 a tracking system is provided. The tracking system is comprised of a 
5 source of energy such as, for example, a light. A passive beacon such as, for example, a retro- 
reflective dot or an RFID tag is located on or associated with the item that is activated by the 
source of energy or the passive beacon reflects energy from the source of energy. Two or more 
fixed detectors are provided with each having a defined field of view that are each capable of 
detecting energy transmitted or reflected from the passive beacon if the passive beacon is in the 

10 fixed detector's field of view. A passive beacon location tracking computer is in communication 
with the two or more fixed detectors. The passive beacon location tracking computer knows the 
location of each fixed detector relative to the other fixed detectors and the passive beacon 
location tracking computer is able to compute the location of the passive beacon from the energy 
received by the two or more fixed detectors from the passive beacon as the location of the item 

15 changes. 

[0126] In Step 2506, information about an item's location is provided to the local computer 
from the tracking system so that the local computer can determine what items are in the data 
acquisition and display device's field of view. 

[0127] In Step 2508, information about those items in the field of view of the data 
20 acquisition and display device is displayed in the see-through display such that the instructions 
and information appear proximately superimposed on the items. The process ends at Step 2510. 
[0128] FIG. 26 is a flowchart that describes a method of computing the orientation and 
position of a wearer of a data acquisition and display device in an embodiment of the invention. 
The process begins at Step 2600. In Step 2602, two or more unique active beacons having 
25 known locations relative to one another are provided. In Step 2604, a data acquisition and 
display device having a beacon detection device with a defined field of view is provided. At 
Step 2606, two or more unique active beacons within the beacon detection device's field of view 
are sensed by the beacon detection device. At Step 2608, the location of the data acquisition and 
display device relative to the known location of the two or more unique active beacons within the 
30 field of view of the beacon detection device is determined. The process ends at Step 2610. 
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[0129] Embodiments of the invention may be used in various applications in parcel and mail 
sorting and processing. For instance, in one embodiment, certain people with a 
sorting/processing facility may be able to see different information about items than what other 
wearers of a data acquisition and display device may be able to see. Examples include high- 
5 value indicators, hazardous material indicators, and items requiring special handling or 
adjustments. Security may also be facilitated by the use of embodiments of the system as items 
are constantly tracked and their whereabouts recorded by the tracking system as they move 
through a facility. And, as previously described, embodiments of the invention may be used to 
track item flow through a facility such that the flow may be enhanced or optimized. 

10 [0130] Embodiments of the invention may also be used in applications other than parcel or 
mail sorting and processing. Many applications involving queues and queuing may make use of 
embodiments of the system. For instance, air traffic controllers managing ground traffic at an 
airport may have information about flights superimposed proximately about or over the actual 
airplanes as they are observed by a controller wearing a data acquisition and display device. 

15 Similarly, train yard operators and truck dispatchers may have information about the trains or 
trucks, their contents, etc. displayed on the actual trains and/or trucks. Furthermore, sorting 
facilities other than mail and parcel sorting facilities may make use of the embodiments of the 
invention. For instance, embodiments of the invention may be used in the sorting of baggage at 
an airport whereby sorting instructions will be displayed to sorters wearing a data acquisition and 

20 display device. 

[0131] Complex facility navigation and maintenance activities may also make use of 
embodiments of the invention. A wearer of a data acquisition and display device may be able to 
see instructions guiding them to a particular destination. Examples include libraries, 
warehouses, self-guided tours, large warehouse-type retail facilities, etc. Routine maintenance of 

25 apparatuses may be improved by having maintenance records appear to the wearer of a data 
acquisition and display device when the wearer looks at the device in question. 
[0132] Many modifications and other embodiments of the inventions set forth herein will 
come to mind to one skilled in the art to which these inventions pertain having the benefit of the 
teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to 

30 be understood that the inventions are not to be limited to the specific embodiments disclosed and 
that modifications and other embodiments are intended to be included within the scope of the 
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appended claims. Although specific terms are employed herein, they are used in a generic and 
descriptive sense only and not for purposes of limitation. 
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